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ABSTRACT 

The 39-day Summier Academic Skills Achievement Program 
in Franklin County, Ohio, was designed to increase the basic reading 
and mathematics skills of persons of at least high school age. It 
enabled clients to ear^i a full credit and up to $200 ($3.35 an hour) 
for successfully completing the program. Among the major findings of 
an evaluation of the program are the following: (1) a total of 216 
clients were enrolled, and the average daily attendance was 167.8; 
(2) the age of the clients ranged from 14 to 20 with a median age of 
16; (3) about half the clients were males, half females; (4) 
approximately 87 percent of the clients were black, and 11 percent 
were non-minority; (5) data from the 137 clients who took a reading 
or mathematics pretest and posttest, and who attended 80 percent of 
the program, revealed that there was a statistically significant 
improvement in their reading comprehension and mathematics 
computation; (6) the median grade equivalent of reading pretest 
scores was 7.3; of posttest scores, 8.7; (7) of the 136 clients in 
the evaluation sample, approximately 46 percent gained one or more 
years in reading; (8) the median grade equivalent of mathematics 
pretest scores was 7.4, and the median posttest scores, 8.5; and (9) 
of the 121 clients in the evaluation sample, approximately 58 percent 
gained one or more years in mathematics. Due to several problems it 
was not possible to collect posttest data on non-client individuals 
who served as controls; therefore it was impossible to determine the 
degree to which all the positive change could be attributed to the 
program. Data are presented on 24 tables and figures. (Author/BJV) 
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FINAL EVALUATION REPORT 
SUMMER ACADEMIC SKILLS ENHANCEMENT PROGRAM 
SUMMER 1987 

ABSTRACT 



Program Description ; The Summer Academic Skills Achievement Program was funded 
by the Private Industry Council of Franklin County through the Job Training 
Partnership Act. The purpose of the program was to increase the basic reading 
and mathematics skills of persons of at least high school age. The 39-day 
summer instructional program featured the Houghton Mifflin New Directions 
Reading Series and the Houghton Mifflin Individualized Computation Skills 
Series. Both homogeneous and heterogeneous class groupings were used as 
appropriate. 

Clients who participatecJ, in the program could earn a full credit for 
successfully completing both the reading and mathematics parts of the program. 
Clients earuad $3.35 per hour training allowance and could earn up to $200 for 
successfully completing the program. 

Except for a full-time organizer /director, the program consultant, 
counselor, and tearhers served on a part-time bases. The staff provided 
services to the 216 clients enrolled in the program and carried out 
non-instructional activities with non-clients who were to serve as a control 
group. 

Evaluation Questions; The following evaluation questions were specified in the 
evaluation design of the program. 

1. Were there statistically significant differences in gains from 
pretest to posttest between control and client individuals on the 
Reading Comprehension and Mathematics Computation Tests of the 
Comprehensive Tests of Basic Skills (CTBS)? 

2. Were there changes from pretest to posttest in the number of 
clients who exhibited mastery, partial mastery, and non-mastery 
of the 11 objectives measured by the two tests of the CTBS? 

3. What were the distribution of changes in grade eqdvalents and 
normal curve equivalents in Reading Comprehension and Mathematics 
Computation of clients from pretest to posttest on the CTBS? 

4. What were the demographic characteristics and attendance rates of 
clients? 

Evaluation Design; The evaluation design called for administering screening 
tests to approximately 3,000 low achieving individuals between the ages of 
14-21 years. The screening tests to be used were the Reading Comprehension and 
the Mathematics Computation Tests of the Comprehensive Tests of Basic Skills 
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(CTBS). Level H, Form V of the CTBS was to be used. Based on the results of 
the screening tests, the lowest scoring individuals were to be randomly 
selected for placement. in either the client group or the non-client group. The 
latter group would serve as a control group for the clients who would be served 
in the program. After the groups had been established and prior to the start 
of the program, both groups would be pretested with an alternate form (Form U) 
of the CTBS. At the end of the program, both groups would be posttested with 
the same tests. Client attendance, enrollment, and demographic information 
would be collected on a Pupil Census Form completed by program personnel. 

Due to several problems it was not possible to carry out the screening 
procedure as planned nor to collect posttest data on non-client individuals. 
Therefore, the client and non-client comparisons called for in the evaluation 
design could not be carried out. 

Major Findings; A total of 216 clients were enrolled in the program. The 
average daily membership of the program was 213.0. The average daily 
attendance of the program was 167.8. The chronological age of the clients 
ranged from 14 to 20 with a median age of 16. While the gender of the client 
group was approximately half males and half females, about 87% of the clients 
were black and 11% were non-minority. 

The program evaluation sample was composed of the 137 clients who took a 
reading or mathematics pretest and a reading or mathematics posttest, and 
attended 80% (i.e., 31 days) of the 39-day instructional program. Analyses of 
the pretest-posttest data for these clients revealed that there was a 
statistically significant (£ <.001) increase in the number of items correctly 
answered for both Reading Comprehension and Mathematics Computation. 

Analysis of the change in the number of clients mastering the six reading 
anu five mathematics objectives assess by the CTBS showed that there was 
statistically significant (£ <.05) improvement for each objective. 

0 

An analysis of reading pretest scores showed that the median grade 
equivalent was 7.3, and the median posttest score was 8.7. Of the 136 clients 
in the evaluation sample, approximately 46% gained one or more years in 
reading. The same analysis for mathematics showed that the median pretest 
score was 7.4, and the median posttest score was 8.5. Of the 121 clients in 
the evaluation sample, approximately 58% gained one or more years in 
mathematics. In summary, the analyses used to answer Evaluation Questions 1-3 
showed that clients made substantial gains in both reading and mathematics. 

S ummar y /Re commenda t i ons 

The clients did show substantial growth in reading and mathematics. 
Unfortunately, the loss of a comparison group made it impossible to determine 
the degree to which all the positive change could be attributed to the program. 

Many of the constraints to the first-year operation of the program could 
be overcome by staring client recruitment, teacher selection, and general 
program organization in January. In general, an increase in process and 
product evaluation data could help identify methods of refining a program that 
has shown potential for improving the basic skills of a client group with a 
demonstrated need for such a program. 
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SUMMER ACADEMIC SKILLS ENHANCEMENT PROGRAM 
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Program Description 

The Summer Academic Skills Achievement Program was funded by the Private 
Industry Council of Franklin County through the Job Training Partnership kct. 
The purpose of the program was to increase the basic reading and mathematics 
skills of persons of at least high school age who demonstrated a need to 
improve their skills in these areas • 

The program was modeled on the Columbus Public Schools Summer School 
Program. Clients of the program attended classes from 8:10-11:50 each morning 
and received approximately three and one-half hours of instruction each day* 
The program ran for ei'ght weeks during the Summer of 1987. 

The Houghton Mifflin New Directions Reading Series was used for reading 
instruction. It is a high interest - low reading ability type series* The 
CTBS Locator Test was used to establish classes of clients of homogeneous 
reading skills. The Houghton Mifflin Individualized Computation Skills Series 
was used for mathematics instruction. This is a completely individualized 
instructional program. Unlike the reading classes » heterogeneous grouping was 
used to set up the mathematics classes. 

Clients were recruited by the Private Industry Council through media ads, 
posters, regular school counselors, and mass meetings at a local recreation 
center. Clients were told that they would earn a full credit for passing the 
reading and mathematics courses. One-half credit would be earned for passing 
either one of the two courses. In addition clients would earn a training 
allowance of §3.35 per hour. They were also told that they would earn §100 for 
each one-half credit they earned. The recruitment procedures resulted in an 
enrollment of 216 clients for the prograffi. 

The program staff included one part-time consultant, an 
organizer/director, a coordinator, 14 part-time teachers, and one part-time 
counselor. All staff members were certified teachers. They all received one 
week inservice before the program began. In addition, six part-time teachers 
were hired to administer and score tests given prior to the 39-day 
instructional program. 

Evaluation Questions 

Evaluation Questions 1 : Were there statistically significant differences in 
gains from pretest to posttest between control and client individuals on the 
Reading Comprehension and Mathematics Computation Tests of the Comprehensive 
Tests of Basic Skills (CTBS)? 

Evaluation Question 2 : Were there changes from pretest to posttest in the 
number of clients who exhibited mastery, partial mastery, and non-mastery of 
the 11 objectives measured by the two tests of the CTBS? 
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Evaluation Question 3 : What were the distribution of changes In grade 
equivalents and normal curve equivalents In Reading Comprehension and 
Mathematics Computation of clients from pretest to posttest on the CTBS? 



The evaluation design called for the administration of screening tests to 
approximately 3,000 low achieving individuals between the ages of 14-21 years. 
The screening tests to be used were the Reading Comprehension and the 
Mathematics Computation Tests of the Comprehensive Tests of Basic Skills (CTBS; 
1981). Level H, Form V of the CTBS was to be used. Based on the results of 
the screening tests, the lowest scoring individuals would be randomly selected 
for placement in either the client group or the non-client group. The matter 
group would serve as a control group for the clients who would be served in the 
program. After the groups had been established and prior to the start of the 
program, both groups would be pretested with an alternate form (Form U) of the 
CTBS. At the end of the program both groups would be posttested with the same 
tests. 

The Reading Comprehension Test is composed of 10 passages of text that the 
examinee is asked to read silently and answer a total of 45 multiple choice 
items. The 45 item are <^ivlded into six objectives. The Mathematics 
Computation Test is composed of 40 multiple choice items that are dlvldec into 
five objectives. The six reading and five mathematics objectives are 
delineated in the Major Findings section of this report. In order to achieve 
an objective, the examinee must correctly answer approximately three-fourths of 
the items for that objective. Possible objective mastery scores are: mastery, 
partial mastery, and non-mastery. 

The tests, which were machine scored by the test publisher, yield a 
variety of scores which Include: number of items correct, national percentiles, 
normal curve equivalents, grade equivalents, scale scores, and objective 
mastery scores. Since the Irregular nature of the educational background of 
the program participants made grade level assignments somewhat arbitrary, and 
since national percentiles and norm curve equivalents are determined by the 
combination of the number of correct items, grade level, and time of the school 
year the test was taken, it was not possible to use these two types of derived 
scores in the analyses. Derived scores that are dependent only on the number 
of items correct were used in the analyses. These Include grade equivalents 
and objective mastery scores. The number of items correct on the pretest and 
the posttest was also analyzed. 

The Irregular nature of the educational background of program participants 
along with their anticipated low skill level in reading and mathematics 
contributed to the decision by program personnel to use a relatively low level 
of the CTBS. Level H was used for both the pretest and the posttest. This 
level is designed for pupils through grade eight. It was thought that a more 
difficult level of the test would produce invalid results due to the fact that 
examinees would have a tendency to guess at items that were too difficult for 
them. A later examination of the raw score distributions for both reading and 
mathematics revealed that the level of test difficulty was a good fit for the 
program examinees. 



Evaluation Question 4 : 
attendance rates of clients? 



What were the demographic characteristics and 



Evaluation Design 
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The locally constructed Pupil Census Form was used to collect attendance, 
enrollment, and demographic information on each clients The Pupil Census Form 
was completed by project personnel at the conclusion of the program. 

In May 1987 the recruitment of client and non-client participants for the 
screening phase of the program was initiated. This phase was supposed to 
result in the collection of test data that could be used in the assignment of 
individuals to the client and non-client groups • However, it was found that 
recruitment efforts were not as successful as anticipated and there was 
insufficient time to achieve the recruitment of the desired number of pupils • 
However 216 pupils were enrolled in the program. 

Constraints were also encountered in collecting posttest data on 
non-clients. The plan called for posttesting non-clients at their work sites. 
However, due to a lack of funding, the work sites were closed prior to the end 
of the program. Therefore, non-clients were not available for the completion 
of posttest activities. 

Project personnel were able to collect pretest data on 828 individuals. 
Of this number 306 were assigned to the client group and 522 to the potential 
non-client group. At the end of the program in August 1987, posttest data were 
collected on 160 client pupils. The following section. Major Findings, gives 
detailed information on the results of the analyses that could be carried out 
given the data collection constraints experienced by program personnel. 



Major Findings 

The program evaluation design called for the collection and analysis of 
data to answer four evaluation questions. Since Evaluation Question 4 called 
for the presentation of demographic characteristics and attendance rates of all 
clients, the results for this question are presented first in this section of 
the report. 

Demographic Characteristics and Attendance Rates 

Evaluation Question 4 ; What were the demographic 
characteristics and attendance rates of clients? 

There were 216 clients enrolled in the 39-day instructional program. 
Client days of enrollment ranged from 30 to 39 with a median of more than 38 
days of enrollment. Average daily membership for the program (total days of 
enrollment divided by 39 program days) was 213.0. Client days of attendance 
ranged from zero to 39 with a median of 34 days. Average daily attendance 
(total days of attendance divided by 39 progran» days) was 167.8. The 
chronological age of the clients ranged from 14 to 20 years with a median age 
of 16. Assigned grade level ranged from 8 to 12 with a median of 10. Table 1 
contains the ethnic group and gender of the clients for whom these data were 
reported. Table 2 contains the rate of attendance (attendance divided by 
enrollment) for each demographic group. These data show that, for groups with 
eight or more pupils, non-minority clients had better attendance than black 
clients with black males having the lowest attendance rate. 
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As was detailed in the previous section oJ this report regarding the 
evaluation design, it was not possible to collect pretest-posttest data on both 
client and non -client groups. However, Pupil Census Form (PCF), pretest, and 
posttest data were collected on 155 clients* Except for attendance, the 
descriptive statistics for this group were quite similar to those presented in 
the previous paragraph for all clients enrolled. Attendance was considerable 
higher for the group that had PCF, pretest, and posttest data with the average 
being 4.4 days more than for the total client group. The 4.4 days represents 
about 11% of the 39-day instructional program. 



Table 1 

Ethnic Group and Gender of Clients 
Enrolled in the Program 



Ethnic 



'Gender 



Female 



Male 



Total 



N 



7. 



Non-Minority 


8 


36.4 


14 


63.6 


22 


10.9 


Black 


90 


51.7 


84 


48.3 


174 


86.6 


Asia American 


2 


50.0 


2 


50.0 


4 


2.0 


American Indian 


0 


0.0 


1 


100.0 


1 


.5 


Total 


100 


49.8 


101 


50.2 


201 


100.0 



Table 2 

Attendance Rates of Clients 
Enrolled in the Program 



Ethnic 



Total 



Gender 



Female 



Male 



N Rate 



100 85.5 



101 78.7 



Total 



N 



Non-Minority 


8 


87.9 


14 


88.2 


22 


88.1 


Black 


90 


85.4 


84 


77.5 


174 


81.6 


Asia American 


2 


80.8 


2 


54.8 


4 


68.2 


American Indian 


0 


0.0 


1 


82.1 


1 


82.1 



201 82.1 
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To be included in the pretest-posttest analyses used to answer Evaluation 
Questions 1 through 3, clients had to be in attendance at least 80Z of the 39 
instructional days. This gave assurance that the program had an opportunity to 
benefit the client. The 80Z criterion represents approximately 31 days. Of 
the 155 clients who .had a PCF, pretest, and posttest data, 137 attended at 
least 31 days. These 137 clients comprised the program evaluation sample which 
is the focus of the remainder of this section regarding Major Findings. 

The days of enrollment of the clients in the evaluation sample ranged from 
34 to 39 with a median of more than 38 days« Average daily membership for this 
group was 136.2 for the 137 clients. The days of attendance of the clients in 
the evaluation sample ranged from 31 to 39 days with a median of 36 days. 
Average daily attendance for the group was 125.9 for the 137 clients. The 
chronological age of the clients in the evaluation sample ranged from 14 to 19 
with a median of approximately 16 years. Assigned grade level ranged from 8 to 
12 with a median of 10. Table 3 contains the ethnic group and gender of 
clients in the evaluation sample. 



Table 3 

Ethnic Group and Gender of Clients 
in the Evaluation Sample 



Gender 

Ethnic Female Male Total 



Group N % N % N Z 



Non-Minority 


5 


35.7 


9 


64.3 


14 


10.3 


Black 


70 


58.8 


49 


41.2 


119 


87.5 


Asia American 


1 


1.3 


1 


50.0 


2 


1.5 


American Indian 


0 


0.0 


1 


100.0 


1 


.7 



Total 76 55.9 60 44.1 136 100.0 



Reading and Mathematics Achievement Results 

As was noted earlier, the program personnel were unable to collect 
pretest-posttest data on control individuals. Therefore, the analyses for 
Evaluation Questions 1-3 were limited to the pretest and posttest performance 
of '^lient in the evaluation sample. 

Since the irregular nature of the educational background of the program 
participants made grade level assignments somewhat arbitrary, and since 
national percentiles and normal curve equivalents are determined by the 
combination of the number of correct items, grade level, and time of the school 
year the test was taken, it was not possible to use these two types of derived 
scores in any of the analyses. Derived scores that are dependent only on the 
number of items correct were used in the analyses. These include grade 
equivalents, and objective mastery scores. The number of items correct on the 
pretest and the posttest was used in several analyses. The number of items 
correct on a test is generally considered to be an equal interval scale 
appropriate for use with parametric statistics. 
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The grade equivalent has a serious limitation in doing pretest-post test 
comparisons* The grade equivalent is not an equal unit of measurement* The 
implication of this is that the distance between any two grade equivalents is 
not necessarily equal to the distance between any other two grade equivalents. 
Therefore, arithmetic operations should not be done with grade equivalents. 
They are used in this report as a general point of reference tc inhere clients 
in the evaluation sample scored in terms of grade placement. 

Evaluation Question 1 : Were there statistically 
significant differences in gain scores from pretest to 
post test between control and client individuals on the 
Reading Comprehension and Mathematics Computation Tests of 
the CTBS? 

Since data were not collected on a control group, analyses were carried 
out to determine if there were statistically significant differences from 
pretest to posttest for client individuals. Table 4 contains the results of 
the t^-tests for correlated data for the Reading Comprehension and Mathematics 
Computation Tests. The differences between the number correct on the pretest 
and the posttest were statistically significant (£ <.001) for both the reading 
and mathematics tests. The increase in the number of correct items for 
mathematics was substantial and approximately twice as great as that for 
reading. 

Figure 1 shows the number correct for the pretest plotted against the 
posttest for Reading Comprehension. Clients' scores below an imaginary line 
running from the lower left corner to the upper right corner of the plot 
represent clients whole scores were higher on the posttest than the pretest. 
Figure 2 shows the same information for Mathematics Computation. Of the 136 
pupils represented in Figure 1, 101 (74.3%) had more reading items correct on 
the posttest than on the pretest. Of the 121 pupils represented in Figure 2, 
104 (86.0%) had more mathematics items correct on the posttest than on the 
pretest. 
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Table 4 

Pretest-Posttest Analysis 
for Number of Items Correct 



Pretest Posttest Change 

No* of Standard Standard Standard £ Degrees of 

1 Items N Mean Deviation Mean Deviation Mean Deviation Value Freedom 



Reading 

Comprehension 45 136 27.4 9.2 30.6 10.2 3.2 5.5 6.8*** 135 



Mathematics 

Computation 40 121 19.4 8.9 25.8 10* 2 6.5 6.2 11.6*** 120 



*** £ <.001 
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Except for gender, there were not enough clients to permit analysis by 
demographic variables^ Subsequent analysis by gender did not show 
statistically significant differences between males' and females' 
pretest-posttest change scores for either reading or mathematics. 

Evaluation Question 2 ; Were there changes from pretest to 
post test in the number of clients who exhibited mastery, 
partial mastery, and non-mastery of the 11 objectives 
measured by the two tests of the CTBS? 



The following are the Six Reading Comprehension Test objectives and, in 
parentheses, the number of items measuring each objective. 

Objective 30 - The student will extract details from a passage to answer who, 
want, where, or when questions (9). 

Objective 31 - The student will analyze the feelings, traits, or motives of 
characters in a passage (5)« 

Objective 32 - The student will identify the main idea, the author's purpose or 
v5.ewpoint, or the tone and mood expressed in a passage (5). 

Objective 33 - The student will draw conclusions from or recognize 
cause-and-ef feet relationships in a passage (8). 

Objective 34 - The student will differentiate between reality and fantaey, 
between fact and opinion, or between forms of writing (9). 

Objective 35 - The student will recognize techniques of persuasive writing or 
figurative writing (9). 

The following are the five Mathematics Computation Test objectives and, in 
parentheses, the number of items measuring each objective. Eight of the 40 
items on the math test do not contribute to the objective mastery levels. 

Objective 58 - The student will add decimals or fractions (8). 

Objective 60 - The student will subtract decimals or fractions (8). 

Objective 62 - The student will multiply decimals or fractions (6). 

Objective 64 - The student will divide decimals or fractions (6). 

Objective 65 - The student will solve computation problems involving integers 
(4). 

As indicated in the evaluation question, there are three levels of 
objective mastery (i.e., non-mastery, partial mastery, and mastery). Also, if 
a pupil does not respond to a sufficient number of items for an objective, that 
objective is scored as "omitted." This generally occurs when a pupil does not 
respond to a series of items that appear toward the end of a test. Because of 
the very small number of "omits", they were excluded from the analyses. 
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Objective mastery scores ars provided by the test publisher according to a 
sophisticated scoring method called Item Response Theory (IRT)« This method 
takes into account the level of difficulty of the items a pupil answers 
correctly for purposes of assigning an objective mastery score. As a rule of 
thumb, the pupil must correctly answer about three-fourths of the items for a 
given objective in order to obtain a score indicating mastery of the objective. 

Figures 3-13 give a crosstabulation of the pretest-posttest mastery scores 
for each of the 11 objectives. In these figures the pretest mastery level are 
given down the left side. The frequency and percent of change from pretest to 
posttest are given in the cells of each crosstabulation. An analysis of 
non-mastery pretest scores showed that for reading the greatest increase in the 
percent of pupils moving from non-mastery to either partial mastery or mastery 
was for Objective 33 that measures the pupil's ability to draw conclusions and 
recognize cause arid effect. Of the 22 pupils (16.2%) who were at the 
non-mastery level on the pretest, 10 pupils (45.5%) scored at least at the 
partial mastery level on the posttest. For mathematics, the greatest increase 
in the percent of pupils moving from non-mastery to either partial mastery or 
mastery was for Objective 58 that measures the pupil's ability to add decimals 
and fractions. Of the 64 pupils (52.9%) who were at the non-mastery level on 
the pretest, 34 pupils (53.1%) scored at least at the partial mastery level on 
the posttest. 

Since clients could also move from a pretest score indicating mastery to a 
lower mastery score, the preceding analyses that focused only on clients having 
a non-mastery pretest score can be somewhat misleading. A second analysis was 
done to determine the change in the percent of clients in the non-mastery 
category regardless of their pretest scorec This analysis showed that for 
reading the greatest decrease in the non-mastery category was for Objective 34 
that measures the client's ability to differentiate between reality and 
fantasy, fact and opinion, and between forms of writing. There was a decrease 
of 20 clients (14>7% of the 136 clients) for this objective. For mathematics, 
the greatest decrease in the non-mastery category was for Objective 64 that 
measures the client's ability to divide decimals or fractions. There was a 
de crease of 40 clients (33.1% of the 121 clients) for this objective. 

To determine if the client's improvement in mastery level was 
statistically significant, the Wilcoxon Matched-Pairs Signed Ranks Test was run 
on the 11 objectives. Table 5 shows the statistical significance level for the 
11 objectives, and the number and percent of the times the pretest mastery 
level was lower then, the same as, or higher than the posttest mastery level. 

As the table shows the pretest-posttest changes in the mastery scores for 
all 11 objectives were statistically significant (£ <.05), and 8 of the 11 
objectives were statistically significant at the .001 level. 
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Table 5 



Results of Wilcoxon Match-Pairs Signed Ranks 
Test for Objective Mastery Scores 
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Figure 3# Crosstabulation of pretest-posttest changes of mastery level scores 
for Objective 30 - Extracts details from passage* (Pretest mastery 
level scores appear down the left side and posttest mastery scores 
appear across the top*) 
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Figure 4* Crosstabulation of pretest-posttest changes of mastery level scores 
for Objective 31 - Analyze feelings, traits, motives* (Pretest 
mastery level scores appear down the left side and posttest mastery 
scores appear across the top*) 
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Figure 5» Crosstabulation of pretest-posttest changes of mastery level scores 
for Objective 32 - Identify main idea* (Pretest mastery level scores 
appear down the left side and posttest mastery scores appear across 
the top.) 
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Figure 6. Crosstabulation of pretest-posttest changes of mastery level scores 
for Objective 33 - Draw conclusions; cause and effect. (Pretest 
mastery level scores appear down the left side and posttest mastery 
scores appear across the top») 
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Figure 7. Cross tabulation of pretest-posttest changes of mastery level scores 
for Objective 34 - Differentiate reality from fantasy. (Pretest 
mastery level scores appear down the left side and posttest mastery 
scores appear across the top.) 
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Figure 8« Crosstabulation of pretest-posttest changes of mastery level scores 
for Objective 35 - Recognize techniqu^is of persuasion. (Pretest 
mastery level scores appear down the left side and posttest mastery 
scores appear across the top. ) 
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Figure 9. Crosstabulation of pretest-posttest changes of mastery level scores 
for Objective 58 - Add decimals or fractions. (Pretest mastery level 
scores appear down the left side and posttest mastery scores appear 
across the top* ) 
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Figure 10. Croostabulation of pretest-posttest changes of mastery level scores 
for Objective 60 - Subtract decimals or fractions. (Pretest mastery 
level scores appear down the left side and posttest mastery scores 
appear across the top*) 
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Figure II. Crosstabulation of pretest-posttest changes of mastery level scores 
for Objective 62 - Multiply decimals or fractions. (Pretest mastery 
level scores appear down the left side and posttest mastery scores 
appear across the top.») 
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Figure 12. Crosstabulation of pretest-posttest changes of mastery level scores 
for Objective 64 - Divide decimals or fractions. (Pretest mastery 
level scores appear down the left side and posttest mastery scores 
appear across the top.) 
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Figure 13. Crosstabulation of pretest-posttest changes of mastery level scores 
for Objective 65 - Compute problems with integers, (Pretest mastery 
level scores appear down the left Pide and posttest mastery scores 
appear across the top*) 



Evaluation Question 3 ; What were the distribution of 
changes in grade equivalents and normal curve equivalents 
in Reading Comprehension and Mathematics Computation of 
clients from pretest to posttest on the CTBS? 

As was mentioned earlier, it would have been inappropriate to use the 
normal curve equivalent in the analysis of the data. Therefore, the grade 
equivalent score was used to answer Evaluation Question 3* Figures 14-16 show 
the distribution of the grade equivalent pretest, posttest, and change for 
Reading Comprehension. As Figures 14 and 15 show, the pretest and posttest 
grade equivalents ranged from the lowest possible scores for the test (i.e., 
4.0) to the highest possible score for the test (i.e*., 12»9)t The variability 
of the clients was quite high. The median pretest grade equivalent was 7.3, 
and the median posttest grade equivalent was 8»7. The median change for 
Reading Comprehension was eight months which represents the growth normally 
expected in four-fifths of an academic year. Of the 136 clients, 62 (45. 6Z) 
gained one year or more in the two-month program. The change score 
distribution for reading appears as Figure 16. 

Figures 17-19 show the distributions of the pretest, posttest, and change 
for Mathematics Computation. As Figures 17 and 18 show, the pretest and 
posttest grade equivalents ranged from 4.3 to 12.9. As was true with reading, 
these scores represent the lowest and highest possible scores. Also as with 
reading, the variability in the mathematics skills of the clients was quite 
high. The median pretest grade equivalent for math was 7.4, and the median 
posttest grade equivalent was 8.5. The median change for Mathematics 
Computation was 12 months, which represents the growth normally expected in 1.2 
academic years* Of the 121 clients, 70 (57. 9Z) gained one year or more in the 
two-month program. The change score distribution for mathmatics appears as 
Figure 19. 
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Figure 14« Frequency distribution of pretest grade equivalents for Reading 
Comprehension Test* (The data appear in three sets of four 
columns • The "valve" is the grade equivalent valve.) 
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Figure 15, Frequency distribution of posttest grade equivalent for Reading 
Comprehension Test. (The data appear in three sets of four 
columns. The "valve" is the grade equivalent valve.) 
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Figure 16. Frequency distribution of pretest-posttest grade equivalent change 
for Reading Comprehension Test. (The data appear In three sets of 
four columns. The "valve" Is the grade equivalent valve.) 
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Figure 17, Frequency distribution of pretest grade equivalent for Mathematics 
Comprehension Test. (The data appear in three sets of four 
columns. The "valve" is the grade equivalent valve.) 
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Figure 18. Frequency distribution of posttest grade equivalent for Mathematics 
Computation Test. (The data appear In three sets of four columns. 
The "valve" is the grade equivalent valve.) 
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Figure 19. Frequency distribution of pretesc-posttest grade equivalent changes 
for Mathematics Comprehension Test. (The data appear in three sets 
of four columns. The "valve" is the grade equivalent valve.) 
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Summa ry /Re commend at 1 ons 

Individualized reading and mathematics instructional series were used 
during the 39-day instructional program in an effort to improve the basic 
skills of the 216 clients served by the program* Major incentives that were 
offered to clients include the opportunity to earn a full academic credit, a 
$200 bonus for successfully completing the program, and an hourly rate of pay 
for their participation* 

Despite problems with recruitment and data collection, the evaluation data 
indicated that the program was successful in improving the basic skills of the 
clients in the areas of reading comprehension and mathematics computation* The 
analyses of the pretest-posttest data used to answer the questions in the 
program evaluation design showed that there were statistically significant 
improvement in both reading and mathematics for the number of items correctly 
answered and the number of objective mastered* The analysis of the grade 
equivalent data indicated that the median gain was 8 months in reading and 11 
months in mathematics for the two-month program* 

The following summary statements are based on both formal evaluation 
activities and discussions with program personnel* 

1* The clients did show substantial growth in reading and 
mathematics* To more clearly demonstrate that the growth was 
due to the program, procedures should be established to 
obtain data from a comparison group who did not receive the 
instructional program* Ma:,y of the problems in setting up 
this comparison group could be resolved by staring 
recruitment in January and in setting up alternative 
procedures to collect posttest data* 

2. Many of the other constraints experienced by the program 
during the first year could be resolved by staring activities 
earlier* This includes teacher recruitment and selection* 

3* The number of clients should be increased* Money spent on 
improving basic skills may well have more long-term benefits 
than spending funds on low level jobs* 

In general, if a larger number of clients were served with an adequate 
comparison group, data could be collected to determine curricular areas where 
the program could have the greatett impact* It is possible that the program 
could have even greater benefit, if it were better coordinated with the regular 
school program for those clients still attending school* With the availability 
of more process and product evaluation data, it might be possible to further 
refine a program that has demonstrated potential for improving the basic skills 
of a client group with a demonstrated need for such a program* 
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